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We analyze the effects ol noise correlations in the input to, or among, BCM neurons using the 
Wigner semicircular law to construct random, positive-definite symmetric correlation matrices and 
compute their eigenvalue distributions. In the finite dimensional case, we compare our analytic 
results with numerical simulations and show the effects of correlations on the lifetimes of synaptic 
strengths in various visual environments. These correlations can be due either to correlations in 
the noise from the input LGN neurons, or correlations in the variability of lateral connections in a 
network of neurons. In particular, we find that for fixed dimensionality, a large noise variance can 
give rise to long lifetimes of synaptic strengths. This may be of physiological significance. 
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I. INTRODUCTION 

Receptive field changes in visual cortex, observed in 
the early period of an animal's postnatal development, 
are thought to depend on synaptic plasicity[l],[2j]; the de- 
tailed dynamics of such receptive field modifiability has 
been used to infer the precise form of this plasticity [H, 0] . 
In a classic paradigm, called monocular deprivation, vi- 
sion through one eye is deprived in early development. 
In this case cells in visual cortex tend to disconnect from 
deprived eye[5j- Experiments have shown that if monoc- 
ular deprivation is produced by monocular lid closure 
then a rapid loss of response to the deprived eye oc- 
curs, while if the retina is inactivated by a Tetrodotoxin 
(TTX) injection, significantly less loss is observed 0, 0]- 
These results are consistent with the form of synaptic 
plasticity proposed by Bienenstock, Cooper and Munro 
(BCM)[8[. The BCM theory was originally proposed to 
describe plasticity processes in visual cortex as observed 
by Hubel and Wiesel [5] . One of the main postulates of 
this theory is the existence of a critical threshold (the 
sliding threshold) that depends on past neuronal history 
in a non-linear way. This nonlinearity is necessary to 
ensure stability of the synaptic weights. The main pre- 
dictions of the BCM theory have been confirmed in hip- 
pocampal slices and visual cortex and recently in in vivo 
inhibitory avoidance learning experiments The exten- 
sion of these results to other brain areas, and ultimately 
to the whole brain, is not confirmed but is under active 
study. One motivation for this research is that a pro- 
posed biophysical mechanism for the BCM rule is based 
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on calcium influx through NMDA receptors and phospho- 
rylation state of AMPA receptors and that both recep- 
tors are widely distributed within the brain 10]. This bio- 
physical mechanism is at least partly shared, by the plas- 
ticity rule STDP (Spike-timing-dependent plasticity) fill] 
that describes the synaptic functional change on the ba- 
sis of action potentials timing in connected neurons. The 
main difference between STDP and BCM is that BCM 
is an average time rule and thus does not include micro- 
scopic temporal structures (i.e. it works with rates not 
spikes) [lH, [l3[ . A further analysis that considers the re- 
lation between BCM and STDP rules will be considered 
in a future work. 

The standard theoretical analysis of monocular depriva- 
tion experimental results, according to BCM (see for ex- 
ample |J]) relies on the, seemingly reasonable, assump- 
tion that in the inactivated situation, activity in the lat- 
eral geniculate nucleus (LGN), which is the cortical in- 
put, is reduced compared to the lid closure 
consequence there is a less rapid loss of response to the 
deprived eye. This assumption has been questioned by 
new experimental results. [LJ] 

In this recent study the activity of neurons in LGN 
has been measured during normal vision, when the eye- 
lid of the experimental animals was sutured and when 
TTX was injected into the lid sutured eye. The record- 
ings were made on awake animals while they watched 
movie clips and sinusoidal gratings. A surprising result 
of these experiments is that inactivation does not reduce 
mean activity in LGN compared to lid suture; however 
inactivation produced an increase in correlations between 
different cells within the LGN. Previous experimental re- 
sults in ferret LGNjlol Il6j. and further results in mouse 
LGN[l4| indicate that the activity of nearby cells in LGN 
are correlated, that this activity falls off as a function of 
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the distance between the receptive fields of the cells, and 
that these correlations exist even in the absence of retinal 
activity. 

A recent paper [17| has examined the impact of input cor- 
relations during deprivation experiments and has shown 
that correlations in LGN noise can significantly slow 
down the loss of response to the deprived eye for BCM 
neurons [8], in agreement with experiments. This paper 
also examines the effect of such correlations on a class of 
PCA type learning rules. Thus correlated LGN noise the- 
oretically leads to persistence of synaptic strength for a 
period that depends on the level of noise correlation. As 
a consequence, noise at neuronal level could play a fun- 
damental role in synaptic plasticity. The effect of white 
noise on BCM has been previously studied [l8j|. In this 
paper we show that noise correlations in the input or 
among BCM neurons can be studied by using the eigen- 
value distributions of random positive-definite symmet- 
ric matrices. In a simple but generic case, we explicitly 
compute the distribution by using the Wigner semicir- 
cular law[l9|, |2(|, pointing out the role of correlations 
and the existence of a thermodynamic limit. In the fi- 
nite dimensional case, the analytic results are compared 
with numerical simulations with applications to real con- 
ditions. We also discuss a transition in the eigenvalue 
distribution when the noise level is increased. This phe- 
nomenon implies the existence of states with very long 
lifetimes; these could have physiological significance in 
the development of neuronal systems. 



II. BCM NEURON IN MONOCULAR 
DEPRIVATION 

We briefly review the behavior of a Bienenstock, 
Cooper and Munro (BCM) neuronQ in monocular de- 
privation. Let w be the synaptic weights and x the in- 
put signals received by the synapses, the BCM synaptic 
modification rule has the form 







w = x4>(y,9 m ) 



(1) 



where the modification function <fi(y, 9 m ) depends on the 
neuron activity level y cx x • w (it is assumed a linear pro- 
portionality between the input x and the output y) and 
on a moving threshold 9 m , which is a super-linear func- 
tion of the cell activity history (in a stationary situation 
8 m can be related to the time averaged value < y k > 
where k > 1 of a non-linear moment of the neuron ac- 
tivity distribution)]^. The modification function is a 
non-linear function of the postsynaptic activity y which 
has two zero crossings, one at y=0 and the other at 
y = &m (see fig. [T]). When the neuron activity is above 
the threshold 9 m we have LTP, whereas LTD appears 
when the activity is below the threshold. The nonlinear 
dependence of the threshold on neuron activity solves 
the stability problem of Hebb's learning rule, preventing 
a dynamical divergence of synaptic weights (y = 9 m is 
an attractive fixed point for a stationary input) [J. In 




FIG. 1: The BCM function <j>(y)\ when y is close to zero, we 
can linearize the system {TJ by approximating <f> ~ — ey. 



the simplest form the function is a quadratic function 
4>{y) — y 2 — y9 m and the dynamic threshold 9 m is the 
time-averaged < y 2 > of the second moment of the neu- 
ron activity, which can be replaced by the expectation 
value over the input probability space E(y 2 ) under the 
slow-learning assumption. During MD regime, the in- 
put signal x to the closed eye can be represented by a 
stochastic process with zero mean value and small vari- 
ance (< x 2 1). One can numerically study the effect 
of noise correlation by integrating the system ([1]). Let us 
introduce the input noise 



x(t) = M(t) 



(2) 



where is a stochastic process in R N defined by i.i.d. 
random variables with zero mean value and normalized 
variance. We consider a symmetric matrix A of the form 
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where the coefficients a^- are independent realizations of 
a normalized random variable with zero mean value. The 
covariance matrix of the noise x(t) is 
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Then for a given N, we can vary the correlation among 
the noise components by varying q 6 [0 : 1] keeping 
fixed the noise variance. We have simulated the weight 
dynamics in eq. ([!} by using the modification poten- 
tial <p{y) — y(y — 1) where we set the neuron activity 
y = (x • w)/N to study the limit N 3> 1. In this simple 
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by diagonalizing the covariance matrix C and expand the 
synaptic weights w in the eigenvector base. Then each 
component w\ in this base will evolve according to 



w\(t) = w\(0)e 



where A is the corresponding eigenvalue of the covariance 
matrix. As a consequence when AC 1 we have long time 
non trivial persistent states for the synaptic weights. Ac- 
cording to numerical simulations shown in the fig. [2] this 
is the case for correlated noisy input. In order to per- 
form explicit calculations we assume that the covariance 
matrix is a random positive definite symmetric matrix of 
the form 



FIG. 2: Evolution of the norm of BCM synaptic weights when 
the input is the correlated noise defined by eq. We 
consider increasing values for q: q — 0.25,0.50,0.75,1.00 for 
N — 100. The curves give the evolution of the mean square 
value of the synaptic weights w when the input noise corre- 
lation increases keeping the noise variance fixed (time is in 
arbitrary units). We note the existence of non zero long time 
persistent states when the correlation is high, whereas for low 
correlation value the system tends to the equilibrium state 
w = 0. 



model, if the activity y is below the threshold 9 m = 1 wc 
expect LTD and the synaptic weights w relax towards 
0. To characterize the relaxation process, we compute 
the mean square value of the vector w(i); the results are 
plotted in fig. [5]for increasing values of q. In all cases the 
system ([T]) tends to a stationary solution. For low corre- 
lated noise we recover the equilibrium solution w = as 
expected, but when the input noise is strongly correlated, 
the simulations show the existence of non zero long time 
persistent states for the synaptic weights. Moreover the 
numerical simulations suggest that the persistent states 
correspond to eigenvectors of the noise covariance matrix 
with very small eigenvalues. These results are consistent 
with experimental observations [lij on the activity of neu- 
rons in LGN in the case of monocular deprivation. 
A theoretical approach linearizes the system ([T]) around 
y ~ by considering 9 rn constant. Then eq. ([IJ becomes 
(cfr. fig. [Q) 



w = — ex(w • x) 



so that on average we have 
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where C is the noise covariance matrix. In the case 
of uncorrelated input (Cy = nfSij where nf is the i- 
component noise variance) equation (j4]) becomes 



W; 



and Wi(t) tends exponentially to zero with a scale time 
~ l/(enf). In a generic case the system ([5]) can be solved 



where 



C k i=n 2 {C^+mV kl +0(m 2 )) 
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and Vki is a symmetric random matrix with zero mean 
value, normalized variance and i.i.d. elements. The 
parameter q £ [0, 1] determines the average correlation 
among the inputs and the parameter m 2 is the variance 
of fluctuations of the covariance matrix. The matrix 
has the following eigenvalues 



Xi = 



l + (N-l)q i = l 
l-q i = 2,..,N 



(8) 



Therefore when q — > 1 (i.e. increase of the average 
correlation) the eigenvalues \2,..,n tend to linearly in 
(1— q) and the correlation matrix C° is degenerate. When 
to = 0, the effect of correlations is to increase the decay 
time of tuW oc 1/(1 — q). In the next section we ana- 
lyze the effect of fluctuations 14/ on the eigenvalue dis- 
tributions of the matrix (|6|) in the thermodynamics limit 
N oo. 



III. 



EIGENVALUE DISTRIBUTIONS 



We consider the eigenvalue distributions of the matrix 
Cm (see eq. ([6])) using the unperturbed covariance matrix 
([7]) . Since Chi is a symmetric positive definite matrix we 
can introduce the following representation 

C = (VC0 + mW) 2 = C°+m{VC0W + wVC0) + m 2 W 2 

(9) 

where \/Cq is the symmetric square root of the matrix 
([7]) and and W is a symmetric random matrix with i.i.d. 
elements and zero mean value. By direct calculation we 
obtain the relation between V and W 



v = Vc°w + wVc°, 
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Then W is uniquely determined as a function of V and 
the expectation value of the covariance matrix 

< C>= C° + Nm 2 I N 

In order to control the noise variance in the thermody- 
namic limit, N — > oo, we introduce the scaling parameter 
mo 



m = mo 



N 



(10) 



This is consistent with the noise definition in the numer- 
ical simulations (cfr. eq. ([3])). The y/1 — q dependence 
allows us to control the limit q — > 1 when the unper- 
turbed matrix C° is degenerate. In real systems both q 
and N are fixed by physical conditions and to-o is pro- 
portional to the fluctuations of the covariance matrix. In 
order to study the eigenvalue distribution of the covari- 
ance matrix C we consider the case when W is a Wigner 
matrix [2lj|. Then Wigner's theorem [HI, on the eigen- 
value distribution can be stated as: 



Let Wij be a symmetric N x N random matrix with 
i.i.d. elements whose common distribution has zero 
mean value, normalized variance and is even, then the 
probability distribution of the eigenvalues in the interval 
[y/Na, VNP] 



P N (VNa <p< v N/3) 
is in the thermodynamics limit 
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To treat the problem of characterizing the eigenvalue dis- 
tribution function when the covariance matrix C has the 
form (jnj), we first perform the orthogonal transformation 
O that diagonalizes the unperturbed matrix ([7]). We then 
restrict our analysis to the invariant subspace that cor- 
responds to the degenerate eigenvalue 1 — q. In the ther- 
modynamic limit the eigenvalue 1 + (N — l)q is singular 
and decouples the corresponding invariant subspace. The 
covariance matrix (JSJ has the form 

C' = OCO T = ^ y/T=qI N + too yjlj^lw^ N » 1 



where W = OWO T . The eigenvalue equation is 
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This is equivalent to the condition 

det[(y/Nu.I N - W)] = 



= (12) 



(13) 



where we have the relation 

A = (l + m fi) 2 (l-q) 



(14) 



and /i are the eigenvalues of a Wigner matrix scaled by 
\/~N. As a consequence we have the following Lemma: 



Any eigenvalue A of the matrix ifTT)) can be written in 
the form {73]) where yfN \i is an eigenvalue of the Wigner 
matrix W so that [i 6 (—2, 2) 

The relation (IT41 leads to the existence of two regions in 
the A spectrum according to the sign of mo/i + 1. When 
fi > — 1/mo we invert the relation (fT4")l by taking the 
positive branch of the square root 



moy/1 - q 



(15) 



but when u. < —1/mo we have to consider the negative 
branch 
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(16) 



The value /i = 1/ to is a critical value for the A spectrum 
and we expect a singularity in the eigenvalue distribution. 
The second case occurs only if mo > 1/2, so that there 
exists a critical threshold in the noise level that induces 
a transition in the A distribution. By applying Wigner's 
theorem to obtain the distribution density of fi in the 
thermodynamic limit we obtain 
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so that in the case mo < 1/2 the distribution p(X) follows 
from the definition fp~5]) according to[22l. I23I] 



p(X) = p(p) 



dp 
dX 
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where 



(1 - q)(l - 2m ) 2 < A < (1 - q)(l + 2m ) 2 

In the case mo > 1/2 we split the A distribution in two 
parts: when 

(1 - g)(2m - if < X < (1 - q)(l + 2m ) 2 

p(X) is given by eq. (fTT|). whereas when 

0< A< (l-g)(2m -l) 2 

the distribution is 
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FIG. 3: Plots of eigenvalue distributions p(A) (see defini- 
tion (|17p ) for the correlation matrix (3) using mo = .2 and 
q = .5, .7, .9 respectively the dashed-dotted, the dotted and 
the dashed line. As the average correlation q increases the 
eigenvalue spectrum is squeezed towards the origin. 



In the second case the distribution is continuous for A > 
and singular at A — > 0. This result leads to the possibility 
that the correlation matrix Q has eigenvalues arbitrarily 
close to zero and larger than one (corresponding to anti- 
correlation effects in the noise). The unperturbed corre- 
lation q among the initial processes x(i) introduces what 
is essentially a scaling factor that changes the existence 
interval of the A spectrum. The critical value mo = 1/2, 
corresponds to a critical value of the fluctuation variance 



(cfr. definition ((T 
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(19) 



From a biophysical point of view, the results (JTHJ) implies 
that for sufficiently correlated noise there exist eigenvec- 
tors w( k ' of the synaptic weights that are preserved for 
a very long time. This might be a possible mechanisms 
of maintaining synaptic weights in neuronal systems in 
correlated noisy environments. 



IV. NUMERICAL RESULTS ON FINITE 
DIMENSION COVARIANCE MATRICES 

To study the effects of finite dimensions on the theo- 
retical results (jTT|) and (fl"8f we have performed some nu- 
merical simulations using matrices of dimension N = 10 3 
that might be realistic for a neural network in LGN. In 
fig. [3] we show the eigenvalue distribution (|17p for nio = 
.2 and different values of the correlation (q — .5, .7, .9). 
To see the effect of the transition in the distribution func- 
tion at the critical value (fH)]) . we computed the eigenval- 
ues distribution in the cases q = .5 and m = .4, .6, .8. 
The results are shown in fig. @] 

We observe, as in the case of low correlation, the eigen- 
value distributions become larger than in the correlated 
case and have values beyond 1 when mo increases. When 
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FIG. 4: Plot of the eigenvalue distribution for the correlation 
matrix (3) using q — .5 and mo = .2, .4, .6, .8 from left to right 
and top to bottom. The effect of the transition at the critical 
value mo = .5 is clearly seen: the eigenvalues accumulate at 
the origin. The continuous line is the theoretical distribution 
(|17[) whereas the histograms are numerically computed using 
a random matrix of order N = 10 3 . 



mo — > 1/2 the distribution becomes singular and the dis- 
tribution peak tends to zero, so that there exist eigen- 
vectors with extremely long decay times in eq. 
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V. GENERALIZATION TO A NEURAL 
NETWORK 



For a network of neurons, eq. ([TJ can be generalized 
24] to 

*(*)=x(V,9W) k = l,...,N (20) 

where the index k refers to the fc-th neuron of the network 
and N is the number of neurons. We then have 

N 

j / W=xW.wW+^W ) (21) 
i=i 

where we have introduced lateral connections, L, among 
the neurons. We can have both excitatory or inhibitory 
neural connections according to the sign of Lki ■ In what 
follows we assume that L is a symmetric matrix with 
||i|| <C 1 where || || denotes the usual matrix norm. 
Therefore the neurons form a bidirectional symmetric 
network; we can invert the relation (|2Tj) to obtain 

^^(I-L^xW-wW (22) 
l=i 

For monocular deprivation eq. (|21[) becomes a network 
generalization of eq. (U]) 

wW^-fWxW^) k = l,...,N (23) 

where c( k > are suitable positive constants. By substitut- 
ing eq. (|22|) into (|23|) . we get the linear system 

N 

w« - -e«x« - LJ^xW • w« jfc = 1, AT 

i=l 

(24) 

In what follows we regard the inter-neuron connections 
L, as not- modifiable. Moreover for simplicity we reduce 
external input to one dimension for each neuron: then 
both w( fe ) and x( fc ) are scalar. In order to study the 
dynamical properties of the solutions we average on fast 
scale variations of the noise; the equations (|24l) become 

N 

wtk) = _ e (fc) J-(I - L^Cuw® k = 1, N (25) 

where Cm is the noise covariance matrix between the in- 
puts of the k and I neuron. We consider the connection 
matrix L as a symmetric random matrix, so that accord- 
ing to our hypotheses the matrix (I—L)^ 1 is a symmetric 
positive definite matrix. Then (125[) can be written in the 
form 

N 

^{k) = _ e {h)^l uW {l) k = l,...,N (26) 
1=1 



is still a symmetric positive definite matrix. If the input 
signals are independent (Cm = o\&ki), we obtain 

u,W = -e {k) n 2 k (I - L)fc fc V*> k = 1, TV 

and the ui( fc ) exponentially decay towards zero. Again 
the interesting case is when the covariance matrix is not 
diagonal and the connections L are defined by a random 
matrix; from a biological point of view we are modeling 
a ensemble of neurons which are stimulated by corre- 
lated inputs and have bilateral connections with random 
weights. When L has the form (cfr. eq. (JSJ) 

Lki = C° kl + mV M + 0(m 2 ) (28) 

we can apply the method used for a single neuron to 
study the eigenvalue distribution. But in the network 
case we have a new biophysical interpretation of the crit- 
ical value fj 1Q|) : if number of neurons is sufficiently large 
and/or the fluctuations in the bilateral neural connec- 
tions exceed a threshold, the network is able to develop 
W W eigenvectors with extremely long lifetimes in pres- 
ence of a noisy correlated input. This phenomenon sug- 
gests mechanisms for long network lifetimes in noisy en- 
vironments. The possibility that correlated noise plays a 
role in the dynamics of a neural network has been inves- 
tigated in 25] using different models. 

VI. CONCLUSIONS 

We have analyzed the effects of noise correlations in the 
input to, or among, BCM neurons using the Wigner semi- 
circular law to construct random, positive-definite sym- 
metric correlation matrices and computing their eigen- 
value distributions. In the finite dimensional case, our 
analytic results are compared with numerical simula- 
tions. We thus show the effects of correlations on the 
lifetimes of the synaptic strengths in various visual envi- 
ronments. In the case of a single neuron the noise correla- 
tions arise from the input LGN neurons, whereas the cor- 
relations arise also from the lateral connections in a neu- 
ron network. If the dimensionality of system is fixed, we 
show that when the fluctuations of the covariance matrix 
exceed a critical threshold synapses with long lifetimes 
arise. These results may be of physiological significance 
and can be tested experimentally. 



where 



L k i - {I - L) kl Cki 



(27) 
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strengths in various visual environments. These correlations can be due either to correlations in 
the noise from the input LGN neurons, or correlations in the variability of lateral connections in a 
network of neurons. In particular, we find that for fixed dimensionality, a large noise variance can 
give rise to long lifetimes of synaptic strengths. This may be of physiological significance. 

PACS numbers: 87.18.Tt,05.40.-a,87.18.Sn 
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I. INTRODUCTION 

Receptive field changes in visual cortex, observed in 
the early period of an animal's postnatal development, 
are thought to depend on synaptic plasicity[? ? ]; the 
detailed dynamics of such receptive field modifiability has 
been used to infer the precise form of this plasticity [? ? ]. 
In a classic paradigm, called monocular deprivation, vi- 
sion through one eye is deprived in early development. In 
this case cells in visual cortex tend to disconnect from de- 
prived eye[? ]. Experiments have shown that if monocu- 
lar deprivation is produced by monocular lid closure then 
a rapid loss of response to the deprived eye occurs, while 
if the retina is inactivated by a Tetrodotoxin (TTX) in- 
jection, significantly less loss is observed [? ? ]. These 
results are consistent with the form of synaptic plasticity 
proposed by Bienenstock, Cooper and Munro (BCM)[? 
]. The BCM theory was originally proposed to describe 
plasticity processes in visual cortex as observed by Hubel 
and Wiesel [? ] . One of the main postulates of this the- 
ory is the existence of a critical threshold (the sliding 
threshold) that depends on past neuronal history in a 
non-linear way. This nonlinearity is necessary to ensure 
stability of the synaptic weights. The main predictions 
of the BCM theory have been confirmed in hippocampal 
slices and visual cortex and recently in in vivo inhibitory 
avoidance learning experiments [? ]. The extension of 
these results to other brain areas, and ultimately to the 
whole brain, is not confirmed but is under active study. 
One motivation for this research is that a proposed bio- 
physical mechanism for the BCM rule is based on cal- 
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cium influx through NMDA receptors and phosphoryla- 
tion state of AMPA receptors and that both receptors are 
widely distributed within the brain[? ]. This biophysi- 
cal mechanism is at least partly shared, by the plastic- 
ity rule STDP (Spike-timing-dependent plasticity) [? | 
that describes the synaptic functional change on the ba- 
sis of action potentials timing in connected neurons. The 
main difference between STDP and BCM is that BCM 
is an average time rule and thus does not include micro- 
scopic temporal structures (i.e. it works with rates not 
spikes) [? ? ]. A further analysis that considers the re- 
lation between BCM and STDP rules will be considered 
in a future work. 

The standard theoretical analysis of monocular depri- 
vation experimental results, according to BCM (see for 
example [? ]) relies on the, seemingly reasonable, as- 
sumption that in the inactivated situation, activity in 
the lateral geniculate nucleus (LGN), which is the corti- 
cal input, is reduced compared to the lid closure 
a consequence there is a less rapid loss of response to the 
deprived eye. This assumption has been questioned by 
new experimental results. [? ] 

In this recent study the activity of neurons in LGN 
has been measured during normal vision, when the eye- 
lid of the experimental animals was sutured and when 
TTX was injected into the lid sutured eye. The record- 
ings were made on awake animals while they watched 
movie clips and sinusoidal gratings. A surprising result 
of these experiments is that inactivation does not reduce 
mean activity in LGN compared to lid suture; however 
inactivation produced an increase in correlations between 
different cells within the LGN. Previous experimental re- 
sults in ferret LGN[? ? ], and further results in mouse 
LGN[? ] indicate that the activity of nearby cells in LGN 
are correlated, that this activity falls off as a function of 



